Methods such as posting flyers in public places, placing print ads in newspapers and magazines, and posting online classified ads on Craigslist have been widely used to recruit research subjects. Recently, the rise of social media Websites (e.g., Facebook) and online services such as Google Ads and Amazon’s Mechanical Turk (MTurk) offer new opportunities for researchers to recruit study participants. Although researchers have started to use these emerging methods, little is known about how they perform in terms of cost efficiency and, more importantly, the type of people that they ultimately recruit. Here, we report findings about the performance of four online sources for recruiting participants, in our case, iPhone users: Craigslist, Facebook, Google Ads and MTurk. First, we compare the cost and participant demographics associated with different recruiting sources. Next, we evaluate whether people recruited from different sources behaved differently in our screener survey (a brief online questionnaire to collect participants’ demographic information and to verify they are actually iPhone users). The findings reveal very different performance between two types of online recruitment strategies: those that “pull-in” online users actively looking for paid work (e.g., MTurk workers and Craigslist users) and those that “push-out” a recruiting ad to online users engaged in other, unrelated online activities (e.g., Google ads and Facebook). We find that (1) the pull-in recruiting strategy was more cost efficient (more respondents per dollar) than the push-out approach; (2) participants from the two pull-in sites (Craigslist and MTurk) were predominantly young presumably because the users are relatively young; (3) the two push-out recruiting sources, in contrast, seemed to have reached a more diverse user base. In addition, the pull-in strategy brought in participants who seemed more committed to the task and more willing to disclose personal information in the interview, than respondents attracted through push techniques.